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(1) Real Party in Interest : 

The real party in interest is assignee Educational Testing Service. 

(2) Related Appeals and Interferences: 

No other appeals or interferences exist which relate to the present application or 
appeal. 

(3) Status of Claims: 

Claims 1, 4-5, 7-12, 14, 16, 21-26, 28, and 30 are pending, finally rejected, and 
appealed herein. Claims 2-3, 6, 13, 15, 17-18, 20, 27, and 29 were canceled in a post-appeal 
amendment per 37 CFR 41.33(b)(1) submitted herewith. Claim 19 was canceled and claims 
31-54 were withdrawn in previously entered amendments. 

(4) Status of Amendments: 

Amendments and cancellations made in the amendment after final submitted 
September 18, 2008 were not entered by the Examiner. The claims thus stand as they were 
in the entered amendment submitted on April 30, 2008, except for those claims canceled in 
a post-appeal amendment submitted herewith. 

(5) Summary of Claimed Subject Matter: 

As an initial matter, it is noted that according to the Patent Office, the concise 
explanations under this section are for Board convenience, and do not supersede what the 
claims actually state, 69 Fed. Reg. 155 (August 2004), see page 49976. Accordingly, nothing 
in this Section should be construed as an estoppel that limits the actual claim language. 

As described in the abstract, for example, embodiments of the invention generally 
pertain to automatic evaluation of essays, including detection of overly repetitive word 
usage. More specifically, embodiments of the invention identify a word in the essay, and 
determine at least one feature associated with the word. A probability of the word being 
used in an overly repetitive manner is determined by mapping the feature to a model, the 
model having been previously generated by a machine learning application based on at 
least one human-evaluated essay. When the probability exceeds a threshold, embodiments 
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of the invention annotate the essay to indicate the word is used in an overly repetitive 
manner. 

Claim 1 teaches a method for automatically evaluating an essay to detect a writing 
style error, that comprises electronically receiving an essay on a computer system (page 8 
line 18), and assigning a feature value automatically calculated (page 9 line 14, page 19 line 
1) for each feature of essay text segments (page 9 lines 17-24, page 11 lines 1-2, page 14 line 9 
and 12-15, page 18 line 20). Further, the feature values are stored on a data storage device 
accessible by the computer system (page 21 line 14). Next, the feature values are compared 
with a model configured to identify writing style errors (page 17 lines 9-12). The model 
includes a decision tree to determine a probability associated with a likelihood of a writing 
style error, and the decision tree is generated based on a human evaluated essay (page 16 
lines 13-16 and 21-23). Finally, the method displays an indication of an identified writing 
style error (page 3 lines 23-24 and page 20 line 7). 

Claim 16 teaches a system for automatically evaluating an essay to detect a writing 
style error, that comprises a computer system configured to electronically receive an essay 
(page 18 line 18), and a feature extractor configured to assign a feature value (page 9 line 
14, page 19 line 1) for each feature for essay text segments (page 9 lines 17-24, page 11 lines 
1-2, page 14 line 9 and 12-15, page 18 line 20). Further, a data storage device is connected 
to the computer system and configured to store the feature values for the text segments 
(page 21 line 14). Also, a feature analyzer evaluates the essay for a writing style error by 
comparing the feature values for each text segment with a model (page 17 lines 9-12). The 
model includes a decision tree to determine a probability associated with a likelihood of the 
writing style error, and the decision tree is generated based on a human evaluated essay 
(page 16 lines 13-16 and 21-23). Finally, a display presents the evaluated essay that 
includes an indication of an identified writing style error (page 3 lines 23-24 and page 20 
line 7). 

(6) Grounds of Rejection to be Reviewed on Appeal: 

(a) Whether claims 1, 4, 7-10, 12, 14, 16, 21-24, 26, and 28 have been properly rejected under 
35 U.S.C. § 103(a) as unpatentable over U.S. Patent No. 6,356,864 (Foltz) in view of U.S. 
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Published Application No. 2004/0002994 (Brill), U.S. Published Application No. 
2004/0093567 (Schabes), and, effectively U.S. Published Application No. 2003/0149692 
(Mitchell). 

(b) Whether claims 5 and 30 have been properly rejected under 35 U.S.C. § 103(a) as 
unpatentable over Foltz Brill, Schabes, and Mitchell 

(c) Whether claims 11 and 25 have been properly rejected under 35 U.S.C. § 103(a) as 
unpatentable over Foltz in view of Brill and Schabes and further in view of Mitchell. 

(7) Argument: 

(a) Claims 1, 4, 7-10, 12, 14, 16, 21-24, 26, and 28: 

It is noted that according to the Patent Office, a new ground of rejection in an 
Examiner's answer should be "rare" and should be levied only in response to such things 
as newly presented arguments by an Applicant or to address a claim that the Examiner 
previously failed to address, 69 Fed. Reg. 155 (August 2004), see, e.g., pages 49963 and 
49980. Furthermore, a new ground of rejection must be approved by the Technology 
Center Director or designee and in any case must come accompanied with the initials of the 
conferees of the appeal conference, id., page 49979. Appellants note that the SPE signed off 
on the final rejections. Accordingly it is not expected that reopening of prosecution will 
occur, since the SPE has already had the chance to consider the gravamen of the arguments 
below and has rejected them. 

Appellants respectfully submit that none of the prior art cited by the Examiner 
discloses, teaches, or suggests all the elements of the appealed claims. Therefore, neither 
independent claim 1 nor independent claim 16 is obvious based upon any of the cited art. 
Because claims 1 and 16 are patentable, their dependent claims are patentable at least as 
dependent from patentable base claims, per MPEP § 2143.03 and In re Fine, 837 F.2d 1071 
(Fed. Cir. 1988). Dependent claims 5 and 30, then 11 and 25 are addressed separately 
below. The dependent claims are further patentable based on the additional limitations 
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that are added by each such dependent claim. Accordingly, Appellants respectfully assert 
that the rejections are improper and should be reversed. 

Limitations added to the independent claims in the amendment after final filed on 
September 18, 2008 were not entered by the Examiner. Appellants had amended claims 1 
and 16 to more clearly define the invention and to place the application in better condition 
for allowance or appeal, e.g. to denote that function words of the essay are to be excluded 
when the embodiments of the invention are calculating the feature values. These further 
limitations were originally stated in dependent claim 5, which had been previously rejected 
based on all four of the cited prior art references, Foltz, Brill, Schabes, and Mitchell. 
Appellants assert the Examiner's decision not to enter the amendment after final 
constitutes an assertion as detailed in the advisory action that the content of the 
independent claims is rejected based on all four of the prior art references, and appeal 
accordingly. 

Appellants had further amended claims 1 and 16 to more clearly define the 
invention and to place the application in better condition for allowance or appeal, e.g. to 
denote that misspellings are not among the writing style errors to be detected by 
embodiments of the invention. Appellants acknowledge that spell checkers are known in 
the art. 

Appellants had further amended claim 16 to more clearly define the invention and 
to place the application in better condition for allowance or appeal, e.g. to denote that as 
with claim 1 the model is generated using at least one human evaluated essay by 
embodiments of the invention. This limitation was originally stated in dependent claim 30. 

Appellants had further amended dependent claims 12 and 26 to more clearly define 
the invention and to place the application in better condition for allowance or appeal, e.g. 
to denote that the distance between consecutive text segment occurrences is determined by 
embodiments of the invention by calculating the number of intervening characters. This 
limitation was originally stated in dependent claims 14 and 28. 

Appellants acquiesce to all of the unentered limitations to the claims in the 
amendment after final, and request that the Board consider amendments not entered as the 
subject of any explicit statements of how a claim on appeal may be amended to overcome a 
specific rejection, per 37 CFR 41.50(c). 
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(b) Claims 5 and 30: 

Appellants note that Mitchell extracts nouns, verbs, modifiers, prepositions, 
adjectives, and adverbs for example, and then submits data representations of such 
constituent parts of each student answer to semantic analysis. See for example paragraphs 
[0026] and [0048]-[0054] of Mitchell. Mitchell not only fails to teach or suggest but 
actually teaches away from not considering the function words of the essay, as taught and 
presently claimed in the present invention. In paragraph [0031] of the published version of 
the present patent application (U.S. Patent Application Publication 2004/0194036), 
corresponding to paragraph [0030] on page 9 of the as-filed application, the inventors state 
"Prior to generating the vector files, function words such as prepositions, articles, and 
auxiliary verbs, may be removed . For example, the function words (the, that, what, a, an, 
and, not) have been empirically found to increase the complexity of the analysis without 
contributing to the reliability of the result." (emphasis added). 

The Examiner's assertion that in Mitchell "certain words of the essay are not 
considered by the assessment tool since they are altered to the reduced variant form" is not 
entirely correct; such words are converted to the reduced variant form specifically so they 
can indeed be considered by Mitchell, in contrast to the claimed embodiments of the 
present invention which intentionally do not consider them. Appellants amended the 
independent claims in the unentered amendment after final to clarify that functional words 
are excluded from consideration by embodiments of the present invention, to distinguish 
such exclusion from merely altering words to a root form that then is considered. That 
limitation remains in dependent claim 5, thus it is separately appealed. 

(c) Claims 11 and 25: 

Dependent claims 11 and 25 warrant particular discussion and separate appellate 
review. As with the other claims previously described, Appellants respectfully disagree 
with the Examiner's assertion that Mitchell cures the shortcomings of the other cited 
references. Appellants assert that Mitchell does not process pronouns (vs. general nouns, 
which are distinct from pronouns) at all, as taught and claimed by the present invention. 
Appellants cite for example paragraph [0015] of Mitchell in toto: "Preferably, the natural 
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language processing parses the mark scheme answer into constituent parts such as nouns, 
verbs, adjectives, adverbs, modifiers and propositions." Appellants note that not only this 
paragraph but all of Mitchell entirely fails to mention "pronouns". 

In summary, as the various embodiments of the present invention include features 
neither taught nor suggested by the cited prior art references, either separately or together, 
the obviousness rejections cannot be sustained. Mere untaught possibilities are insufficient 
to defeat patentability, and a prior art reference must be considered in its entirety, i.e. as a 
whole . W.L. Gore & Associates, Inc. v. Garlock, Inc., 121 F.2d 1540, 220 USPQ 303 (Fed. 
Cir. 1983), cert, denied, 469 U.S. 851 (1984). Therefore, reversal is respectfully requested. 

Conclusion 

For the reasons advanced above, Appellants assert the rejected claims are indeed 
patentable, thus the rejections should be reversed. 
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APPENDIX A - APPEALED CLAIMS 

1. A method for automatically evaluating an essay to detect at least one writing style error, 
comprising: 

electronically receiving an essay on a computer system; 

assigning a feature value for each of one or more features for one or more text segments in 
the essay, wherein the feature values are automatically calculated by the computer 

system; 

storing the feature values for the one or more text segments on a data storage device 
accessible by the computer system; 

comparing the feature values for each text segment with a model configured to identify at 
least one writing style error, wherein the model includes at least one decision tree to 
determine a probability associated with a likelihood of the at least one writing style 
error, and wherein the at least one decision tree is generated based on at least one 
human evaluated essay; and 

displaying an indication of an identified writing style error. 

4. The method of claim 1 wherein the comparison step comprises extracting patterns from 
the feature values, wherein the patterns are based on the presence or absence of features 
associated with each word in the essay. 

5. The method of claim 1, wherein the function words of the essay are not considered by 
the computer system in determining the feature values. 

7. The method of claim 1 wherein the feature values comprise the ratio of the evaluated text 
segment occurrences in the essay to the total number of text segments in the essay. 

8. The method of claim 1 wherein the feature values comprise the average, over all 
paragraphs of the essay, of the ratio of the number of times the evaluated text segment 
occurs in a paragraph of the essay, over the total number of text segments in the paragraph. 
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9. The method of claim 1 wherein the feature values comprise the largest value of the ratio 
of the number times the evaluated text segment occurs in a paragraph of the essay over the 
total number of text segments in the paragraph, wherein the ratio is calculated for each 
paragraph in the essay. 

10. The method of claim 1 wherein the feature values comprise the length, measured in 
characters, of the text segment. 

11. The method of claim 1 wherein the feature values comprise a value indicating whether 
the text segment includes a pronoun. 

12. The method of claim 1 wherein the feature values comprise a value representing the 
interval distance between consecutive text segment occurrences. 

14. The method of claim 12 wherein the distance is determined by calculating the number 
of intervening characters. 

16. A system for automatically evaluating an essay to detect at least one writing style error, 
comprising: 

a computer system configured to electronically receive an essay; 

a feature extractor configured to assign a feature value for each of one or more features for 
one or more text segments in the essay; 

a data storage device, connected to the computer system, configured to store the feature 
values for the one or more text segments; 

a feature analyzer configured to evaluate the essay for at least one writing style error by 
comparing the feature values for each of one or more text segments with a model, 
wherein the model includes at least one decision tree to determine a probability 
associated with a likelihood of the at least one writing style error, and wherein the at 
least one decision tree is generated based on at least one human evaluated essay; and 
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a display for presenting the evaluated essay, wherein the evaluated essay includes an 
indication of at least one identified writing style error. 

21. The system of claim 16 wherein the feature extractor comprises an essay ratio 
calculator configured to generate a value representing the ratio of the number of times the 
evaluated text segment occurs in the essay to the total number of text segments in the essay. 

22. The system of claim 16 wherein the feature extractor comprises an average paragraph 
ratio calculator configured to generate a value representing the average over all 
paragraphs in the essay of the ratio of the number of times the evaluated text segment 
occurs in a paragraph of the essay over the total number of text segments in the paragraph. 

23. The system of claim 16 wherein the feature extractor comprises a highest paragraph 
ratio calculator configured to generate a value representing the largest ratio of the number 
of times the evaluated text segment occurs in a paragraph of the essay over the total 
number of text segments in the paragraph. 

24. The system of claim 16 wherein the feature extractor comprises a length calculator 
configured to generate a value representing the length, measured in characters, of the text 
segment. 

25. The system of claim 16 wherein the feature extractor comprises an identifier to 
determine whether the text segment includes a pronoun. 

26. The system of claim 16 wherein the feature extractor comprises a distance calculator 
configured to generate a value representing the distance between consecutive text segment 
occurrences. 

28. The system of claim 26 wherein the distance between consecutive text segment 
occurrences is measured in characters. 
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30. The system of claim 16 wherein the model is generated using at least one human 
evaluated essay. 
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APPENDIX B - EVIDENCE 

None (this sheet made necessary by 69 Fed. Reg. 155 (August 2004), page 49978). 
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APPENDIX C - RELATED PROCEEDINGS 

None (this sheet made necessary by 69 Fed. Reg. 155 (August 2004), page 49978). 
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